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Abstract. We construct examples of a random walk with palrwlse-lndependent 
steps which Is almost-surely bounded, and for any m and k a random walk 
with fc-wlse independent steps which has no stationary distribution mod- 
ulo 771. 



1. Introduction 



Consider a simulation of a simple random walk on a graph. How will the 
simulation be affected if the source of randomness is not truly random but 
only pseudo random in the following specific sense, the random bits are k- 
wise independent for some k > 1 and not independent as a family? The first 
question to ask is, does the pseudo walk converge to the same stationary 
measure? The most simple graph to consider might be a cycle of size m. 
This suggests the following problem: given a random walk S n = J27=i 
where the Xi are random signs, plus or minus 1 with equal probability, 
and the X{s are k-wise independent, what can be said about the behavior 
of the partial sums and in particular modulo some fixed number m? It 
turns out that there is a fundamental difference between the cases k = 2 
and k > 2. 

Examine first the case k = 2. For this case we shall show 

Theorem 1. There exists a sequence of random variables {X i }°Z 1 taking 
values ±1, pairwise independent, such that S n is bounded almost surely. 

This result should be contrasted against the fact that we know that 
ES n = and ES% = n, just like in the completely independent case. In other 
words, S n for large n must have extreme "fat tail" behavior. Naturally, M := 
max,, S n satisfies EM = oo. The example we will demonstrate is essentially 
the Walsh system, which is pairwise independent. We will discuss this in 
section |2j 

Such behavior cannot occur when k > 4 for the simple reason that in 
this case we know that ES„ = 3rt 2 - 2n and this gives 



Problem 1. Is there a sequence of random variables {X i } c *L l taking val- 
ues ±1 with equal probability 3-wise independent, such that S n is bounded 
almost surely. 

The higher k is, the more moments we know and we can approximate 
the large scale shape of S n . However, this does not necessarily mean we 




n 



VM. 



We could not settle k = 3, 



i 



RANDOM WALKS WITH k -WISE INDEPENDENT INCREMENTS 



2 



can say something about S n mod m. And indeed, in section|3]we show the 
following 

Theorem 2. Let m and k be some natural numbers, and let e > 0. Then 
there exists a sequence of random variables taking values ±1, k- 

wise independent, and a sequence Ij such that 

(1) P (Si } = (m)) >l-e Vj. 

Notice that the requirement that the condition holds only for some Ij 
is unavoidable, since, say, if k > 10m 2 then the distribution of Sj.+iom 2 is 
approximately uniform, since Xi j+ i, . . . ,X Ij+Wm 2 are independent. 

Explicit constructions of fc-wise independent 0- 1 probability spaces and 
estimates on their sizes are the focus of several papers in combinatorics 
and complexity, see e.g. |3| for the first construction and 1 1| for a recent 
discussion with additional references. Sums of pairwise independent ran- 
dom variable were extensively studied, see e.g. O for some interesting 
examples. Thus there are already many interesting examples of pairwise 
independent processes. But it seems behavior modulo m was not studied. 

2. PAIRWISE INDEPENDENT PROCESSES 

We term the construction we use a "gray walk", as it is based on the 
well-known Gray code construction in combinatorics. The nth Gray code 
is a Hamiltonian path on the discrete n-cube, or a listing of all 2" binary 
strings of length n, starting with the string 00. ..0, where every string ap- 
pears exactly once, and any two consecutive strings differ in exactly one 
place. The construction (and hence also proof that this is possible) is 
done recursively, namely: To construct the ?ith Gray code, write down two 
copies of the (n — l)th code, where the order of the strings in the second 
copy is reversed, and add to the first 2"~ 1 strings a zero at the nth place, 
and to the second 2"~ 1 strings a one at the nth place. 

The Gray codes of all orders n can be combined into an infinite Gray 
code, by listing them sequentially for increasing n's, and converting all 
strings into infinite strings by padding with zeros. The first few strings in 
the infinite code are: 





= 000000 . . . 


Ax 


= 100000 . . . 


A 2 


= 110000 


A, 


= 010000... 


A A 


= 011000... 


A 5 


= 111000 


A, 


= 101000... 


A 7 


= 001000 . . . 


As 


= 001100 



To construct the gray walk, we now consider each string A t as specifying 
a finite subset (also denoted AJ of the natural numbers N, where a 1 in 
the jth place signifies that j e A. t , and define 

*i = n & 

jeAi 

where £i, £ 2 , •■• is a seed sequence of independent ±1 variables. It is easy to 
verify that the Xi's are pairwise independent. Therefore to finish theorem 
[l]we only need 
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Proposition. The gray walk S n is bounded almost surely. More precisely 

sup \S n | = 2 mill ^ 1 ^=- 1 }- 1 + 1 = sup S n + 2 = - inf S n 

n n 

Proof. For simplicity we add the element X = 1 , and prove that for S' n = 
YZ=o X i we have BU P S 'n = - inf ^ = 2 min {^ 1: ^=- 1 >- 1 . The recursive defini- 
tion of the Gray code is reflected in the path: Assume we have defined the 
first steps S' , S[, S' 2l S' 2] - 1 _ 1 of the walk, then the next 2 3 1 steps will be 
the previous steps listed in reverse order, and multiplied by the random 
sign £j. This implies that the path up to time 2 J — 1, where j is the first 
value for which & = -1, is 0, 1, 2, 3, 4, 2^, 2^ x - 1, 2^ 1 -2, 3, 2, 1, 0, and 
all the subsequent values are bounded between -2 3 and 2K □ 

Remark. Theorem ^ holds also if one takes the strings A4 in lexicographic 
order, which is simpler and is also the standard ordering of the Walsh 
system. However, we believe the gray walk is interesting in its own right. 
For example, it satisfies that ^j^ 1 = £ju), i.e. the seed sequence is exposed 
directly in the quotients of the X;'s. 

On the other hand, there is a vast body of knowledge about the behavior 
of "weighted walks" with the lexicographical order, which is simply the 
question when 

00 

i=0 

converges where Wi is the Walsh system. The general philosophy is that 
it is similar to the behavior of the Fourier system, with the exception of 
the symmetry of the Fourier system which has no equivalent in the Walsh 
system. Of course, this is very different from the simple behavior of inde- 
pendent variables. See |H) for a survey, or the book |4]. 



3. Proof of theorem [2] 

Before embarking on the general proof, let us demonstrate the case 
m = 4 which is far simpler. Let L > k satisfy L = mod 4. Let be 
a sequence of i.i.d ±1 variables. Next define new variables by conditioning 
the &-s: 

{Xi}:= j& ne„L+4 = lVo = 0,l,... j . 

The fact that L > k clearly shows that the Xi's are fc-independent. The 
fact that JlCaL+b = 1 shows that in each block of size L the number of 
— f's is even, and since 4\L we get that = mod 4 for every block. 

Therefore if we define Ij = jL then the condition in ^ is actually satisfied 
combinatorially and not only in high probability. 

How can we generalize this to to 7^ 4? The remaining ingredient in the 
proof is based on the following well known fact: 

It is possible to construct variables with probability |, i, | given a sequence 
of independent variables with probabilities \,\- 
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The algorithm is simple: take two such variables: if they give 11, "out- 
put" 1, if they give 1,-1 "output" 2 and if they give —1,1, "output" 3. If they 
give —1, —1, take a new set of two variables and repeat the process. The 
output is 1, 2 or 3, each with probability i. The time required for each step 
is unbounded but the probability that it is large is exponentially small. 

The proof below combines these two ideas, which we nickname "gen- 
erating hidden symmetries" and "simulating uniform variables" to get the 
result. 

Proof of theorem^ general m. We may assume without loss of generality 
that m is even (otherwise take 2m instead). Let A = A(fc, e) be some suf- 
ficiently large parameter which we shall fix later. Let L := 2(fc + 1) [Am 2 ] 
where [•] stands, as usual, for the upper integer value. 

Lemma. For every fx = 0,2, ...,m — 2 there exists variables and Y>* with 
the following properties 

(1) L M is a random positive integer, L divides L M and P(L M = aL) < 
2- a ~ 1 e for alia > 1. 

(2) — is a sequence ofk-wise independent variables taking 
values ±1 with probabilities ~. 

(3) { {Vi}°l Lfi+1 \ L^} are Ltd ±1 variables 

(4) = L implies Y%=i Vi = M m od m. 

Proof. Define N := 2 [Am 2 ] . The first step is to divide {±1}^ according to 
the sum modulo m, and trim the resulting sets a little so that they all have 
the same size. Precisely, let 



We note that the distribution of J2i=i ^ modulo m is uniform on the set 
0, 2, . . . , m - 2 with an error of Ce- cX . Therefore 2 N - fA < C2 N -° X . For 
j = 0, 2, . . . , m - 2, let Gj be arbitrary sets satisfying 



and IetS^{±l} w \UGj. 

So far we have constructed some general objects having very little to 
do with probability. Next, let be i.i.d variables taking values in 

±1, and let {Si}^ 1 be a division of the £j's into blocks of size N: Ei := 





(^Ni-N+l, ■ ■ ■ , %Ni)- Let 




We define 
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Properties |3] and |1] are obvious from the construction. Property ^ is a 
direct consequence of the estimate P(Sj e S) < Ce~ cX , if only A is large 
enough. Define A so as to satisfy this condition. Therefore we need only 
prove property |2j 

Let therefore i\ < ■ ■ ■ < i k and Si, ... ,6^ £ {±1} and examine the events 



X:={^=6 u ...,C ik =S k } y 



' BN ~j 



We know that F(X) = 2~ k and we need to show that P(X\y) = 2~ fe . Let b be 
some number and let a c {1, . . . , b — 1} be a set with #er = 6 — (fc + 1) and 
examine the event 

S = S(a, 6) = {& € S -O- £ e ct Vi < b} . 

The point of the proof is that the event y is independent of X n S. This is 
because X depends only on k places, but y depends on k + 1 S t 's each of 
which is distributed uniformly. In other words 

p(y\x n s) = F(y) = — 

m 

or 

P(*n*|»- 3*^2 -P(*ni) 

which gives 

p(A-|;y) =^p(A , n5|y) = ^>(;rn<s) =p(at) = 2" fc . □ 

Returning to the proof of the theorem, we let 

I 1 /(/i,i/)e{0,2,...,m-2}xN 

be an independent family of couples of variables constructed using the 
lemma. We now construct our sequence Xj inductively as follows: assume 
that at the ^th step we have defined X\, . . . ,X p where p = p(y) is random. 
Define /i = p(v) = - J2i=i Xi mod m and then define X p+ %, . . . , X p+L ^, v using 

X p+i = Vi 

and p(y + 1) = p(v) + L^ v . This creates a sequence of ±1 variables. To see 
define r(a) — max{v : p(v) < aL} and properties ^ and |4] of the lemma 
will give that 

(aL \ oc 

]T €i (m) ] < ^ P(p(r(a)) = (a - 6)L and L^W).^) > £ max{6, 2}) 
i=l / 6=1 

OO OO 

H P(i M(r(Q)) ' r(a) = cL) <^2- c - 1 e = e . 

fc— 1 c— max{fc,2} 6,c 

Therefore we need only show that the Xis are /c-wise independent. While 
being strongly related to the A: -wise independence of the yf'" it is not an 
immediate consequence of it because of the dependence between the y? ,v 
and the L^'s. 

We prove that the Xis are fc-wise independent inductively. Let I and 
/ be some integers and assume that we have shown that X n , . . . , X im are 
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independent for every to < I and for m = I and i\ < I. Let / = i\ < 12 < ■ ■ ■ < 
ii and let Si, . . . , Si e {±1}. Examine L ' 1 and define 

n = n(L aA ) = #{to : i m < L ' 1 } . 

The induction hypothesis, together with the independence of the various 
YWs shows that X in+1 , . . . ,X H are i.i.d ±1 variables: if n > then this 
follows from the induction hypothesis for m' := m — n while if n = then 
it follows from the induction hypothesis for m' = m and I' := I - L ' 1 . 
Property |3] of the lemma shows that y£ i ,...,y i { are i.i.d ±1 variables. 
Below n we have simple equality: 

X; tm — y^ 1 Vra < n. 

Therefore 

PpTi, =6 1) ...,X il =6 l \L°< 1 )=F(y il =8 1 ,...,y il =6 l \L ' 1 ) 
and summing over L ' 1 we get the required result: 

¥(X n =6 1 ,...,X il = SO = F(y n =6 U .. . ,y u = <5 ; ) = 2~ l 
which finishes the proof of the theorem. □ 

Remarks. (1) It is easy to generalize the theorem to force YliLi e i to 
have almost arbitrary distributions. The only obstacle is the even- 
odd symmetry of the walk. Thus if m is odd, any distribution what- 
soever on the congruence classes modulus m might be achieved, 
while if to is even, any distribution supported on the set of even or 
odd congruence classes can be achieved. 

(2) It is easy to see from the proof that L w km 2 log(fc/e). In other 
words, the precision is asymptotically exponential in the stepping 
of the subsequence Ij. We remind the reader a fact that was 
discussed in the introduction: if k > TO 2 logl/e then necessarily 
Ij + i — Ij > cTO 2 logl/e because immediately after Ij we get a se- 
quence of independent variables. 

(3) Another direction in which fc-wise independent random variables 
may be very different from truly independent variables is their en- 
tropy. It is known that the entropy of n such variables may be as 
low as fclogn, see |3) and Om" pairwise example is optimal in 
this respect: n truly random bits create 2™ pairwise independent 
random variables. The example above is far from it: the entropy is 
linear in the number of bits. However, it turns out that it is possi- 
ble to create an example of fc-wise independent variables with low 
entropy satisfying the conditions of theorem |3J The example, while 
amusing, if a bit off topic hence we will skip it. 

Theorem 3. Let m and k be some natural numbers. Then there exists a 
sequence of random variables taking values ±1, k-wise independent, 

and a sequence Ij such that 

h 

e. t = (m)forj sufficiently large with probability 1. 

i=i 
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The proof is similar to that of theorem |2j but using a different L in each 
step. However there are some additional technical subtleties. Essentially, 
if in the main lemma of the previous theorem we defined a variable F M 
where the parameter fx was used to "return to sync" the series if ever the 
congruence is no longer 0, here we would need variables Y^- T where r is 
some additional parameter needed to "return to sync" in the L domain. 
While being only moderately more complicated, we felt it better to present 
the simpler proof of the previous theorem. 

4. Further related problems 

We end by suggesting further problems related to A; -wise independent 
random variables. 

4.1. The random sign Pascal triangle. Let £* fe ,£~ fe be a family of i.i.d 
random (1/2-1/2) signs for — 1 < fe < n + 1. Define random variables X n , k 
for — l<fc<n+lby the recurrence 

-^0,0 = 1) Xn,-1 = 0, X n ^ n+ i = 0, 
X n ,k = ^n-l,k-l X n-l,k-l + C-l,fc^™-l,fc {0 <k <n) 

Problems: Study the behavior of the central coefficient X 2n>n . Find an 
interpretation of the triangle as "percolation with interference" where two 
closed gates cancel each other out ("quantum percolation"?) Study the 
behavior of the non-central coefficients. 

4.2. 2D Percolation. What about other functions of fc-wise independent 
random variables? For example, instead of considering Bernoulli per- 
colation, the random variables determining the configuration are only k- 
wise independent. Here is an "unimpressive" example: fc-wise independent 
bond percolation onannxn(fc + l)- box in I? such that the probability to 
have a crossing of the narrow side is 1. Of course, even in usual percola- 
tion the probability 1 is 1 — e~ cfe so that the improvement is not exciting. 

The construction is as follows: divide into k + 1 disjoint boxes of size 
n x n. Take the usual independent percolation, conditioned such that the 
number of boxes with crossings is odd. This is like conditioning k + 1 
Bernoulli variables to have an odd sum, so that you get a fc-wise inde- 
pendent structure. If the number is odd, then it is non zero. Can it be 
improved, say to guarantee crossing of the n x n-box? 
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